Methods and systems for estimating quality of experience (QoE) parameters of secured transactions

ABSTRACT

An apparatus is provided for estimating one or more quality of experience (QoE) parameters associated with a specific terminal. The apparatus includes a traffic processor configured to acquire a plurality of transactions for providing multimedia content to a specific terminal. At least one of such transactions is a secured transaction. The apparatus further includes a QoE parameter estimator configured to detect a quality level variation event based on the transactions and the sizes of the transactions, and to estimate one or more QoE parameters based on the detection of the quality level variation event.

RELATED APPLICATIONS

This patent application is a continuation of, and claims priority to andthe benefit of U.S. patent application Ser. No. 14/622,139, entitled“METHODS AND SYSTEMS FOR ESTIMATING QUALITY OF EXPERIENCE (QOE)PARAMETERS OF SECURED TRANSACTIONS,” and filed Feb. 13, 2015, thecontents of all of which are hereby incorporated herein by reference intheir entirety for all purposes.

BACKGROUND

The recent few years has witnessed an explosive growth of data trafficin networks, particularly in cellular wireless networks. This growth hasbeen fueled by a number of new developments including faster, smarter,and more intuitive mobile devices such as the popular iPhone® series andthe iPad® series, as well as faster wireless and cellular networktechnologies that deliver throughputs on par or better than fixed linebroadband technologies.

For many people today, a primary mode of access to the Internet is viamobile devices using cellular wireless networks. Websites such asGoogle™ or YouTube™ provide an increasing amount of multimedia contentto mobile devices. For example, Google™ or YouTube™ provide videos usingHTTP live streaming (HLS) protocols. An HLS protocol is an adaptivebit-rate (ABR) type protocol and is one of the protocols for providingmultimedia content to mobile devices. In the past, some of themultimedia content provided using the HLS protocols are encrypted. Inrecent years, however, websites such as Google™ or YouTube™ areincreasingly encrypting multimedia content provided using the HLSprotocols. For example, videos are increasingly transported using securesockets layer (SSL) or transport layer security (TLS) protocols. Anotheradaptive bit-rate type protocol for providing multimedia content tomobile devices is the dynamic adaptive streaming over HTTP (DASH)protocol. Websites can also provide encrypted multimedia content tomobile devices using the DASH protocols.

When encrypted multimedia content are provided to mobile devices using,for example, SSL or TLS protocols, measurement of quality of experience(QoE) parameters can be affected. Quality of experience parametersinclude, for example, user experience indexing (UXI) parameters thatreflect the subscribers' quality of experience of using the mobiledevices. The UXI parameters include metrics such as the total media timeof the multimedia content provided, the video bitrate, and the amount ofvideo stalling. Measurement of such UXI parameters can be used forimplementing traffic management techniques or for reporting purposes.Traffic management is a broad concept and includes techniques such asthrottling of low priority traffic, blocking or time shifting certaintypes of traffic, and traffic optimization. Optimization of web andvideo traffic is a key component in the array of traffic managementtechniques used by wireless operators. Therefore, when the measurementof the QoE parameters are affected by the encryption of the multimediacontent transmitted to mobile devices, the implementation of trafficmanagement techniques can be negatively affected as well.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary network system, consistentwith embodiments of the present disclosure.

FIG. 2 is a block diagram illustrating an embodiment of an exemplary QoEparameter estimator, consistent with embodiments of the presentdisclosure.

FIG. 3 is a flowchart representing an exemplary method for estimatingQoE parameters associated with a specific terminal, consistent withembodiments of the present disclosure.

FIG. 4A is a flowchart representing an exemplary method for identifyingvideo transactions and audio transactions, consistent with embodimentsof the present disclosure.

FIG. 4B is a diagram illustrating exemplary relations between averagetransaction sizes, the sizes of identified video transactions, and thesizes of identified audio transactions, consistent with embodiments ofthe present disclosure.

FIG. 5A is a flowchart representing an exemplary method for detecting aquality level variation event, consistent with embodiments of thepresent disclosure.

FIG. 5B is a diagram illustrating exemplary relations of transactionsizes and the quality level variation, consistent with embodiments ofthe present disclosure.

FIG. 6A is a flowchart representing an exemplary method for processingvideo transactions or audio transactions, consistent with embodiments ofthe present disclosure.

FIG. 6B is an exemplary timing table illustrating the relations betweenthe elapsed times, measured cumulative media times, and actualcumulative media times, consistent with embodiments of the presentdisclosure.

FIG. 7A is a flowchart representing an exemplary method for identifyinga mapping between video transactions and audio transactions, consistentwith embodiments of the present disclosure.

FIG. 7B is an exemplary table illustrating exemplary relations betweentransaction indices, audio transaction timestamps, and video transactiontimestamps, consistent with embodiments of the present disclosure.

FIG. 7C is an exemplary score matrix illustrating exemplary mappingbetween audio transaction timestamps and video transaction timestamps,consistent with embodiments of the present disclosure.

FIG. 8 is a flowchart representing an exemplary method for determiningone or more quality of experience (QoE) parameters, consistent with theembodiments of the present disclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to the exemplary embodimentsconsistent with the embodiments disclosed herein, the examples of whichare illustrated in the accompanying drawings. Wherever possible, thesame reference numbers will be used throughout the drawings to refer tothe same or like parts.

The present disclosure relates to traffic management, and moreparticularly to estimating QoE parameters in secured or unsecuredtransactions for optimization or reporting of multimedia contentprovided to a specific terminal. The estimation of QoE parametersincludes acquiring a plurality of transactions, detecting a qualitylevel variation event based on the plurality of transactions and thesizes of one or more of the plurality of transactions, and estimatingthe one or more QoE parameters based on the detection of the qualitylevel variation event.

Network congestion or overload conditions in networks are oftenlocalized both in time and space and affect only a small set of users atany given time. This can be caused by the topology of communicationsystems. In an exemplary cellular communication system, such as thesystem shown in FIG. 1, the system can have a tree-like topology, with arouter or a gateway being the root of the tree and the mobile basestations being the leaves. This tree-like topology is similar acrosscellular technologies including Global System for Mobile Communication(GSM), Universal Mobile Telecommunications System (UMTS) adoptingWideband Code Division Multiple Access (W-CDMA) radio access technology,CDMA2000, Worldwide Interoperability for Microwave Access (WiMax), andLong Term Evolution (LTE). In a tree-like structure of a wirelessnetwork, the impact of network overload conditions depends on the levelof aggregation in the network where that overload condition occurs. Forexample, an overload condition at a base station level affects onlythose users who are connected to that base station. Therefore, in someembodiments, the adaptive traffic management identifies the aggregationlevel at which an overload condition occurs and then applies trafficmanagement techniques in a holistic fashion across only those users thatare affected by the overload condition.

Adaptive traffic management is an approach wherein traffic managementtechniques such as web and video optimization can be applied selectivelybased on monitoring key indicators that have an impact on the Quality ofExperience (QoE) of users or subscribers. Applying optimization caninvolve detecting the presence of multimedia content in secured orunsecured transactions, classifying multimedia content in thetransactions, and estimating one or more QoE parameters associated witha specific terminal. The detection of the presence of multimedia contentand the classification of the multimedia content are described in moredetail in related U.S. patent application Ser. No. 14/503,274 filed onSep. 30, 2014. In the present disclosure, a subscriber can be a mobileterminal user who subscribes to a wireless or cellular network service.While the subscriber refers to the mobile terminal user or a user of aspecific terminal here, future references to subscriber can also referto a terminal that is used by the subscriber, or refer to a clientdevice used by the subscriber.

FIG. 1 is a block diagram of an exemplary network system. Exemplarycommunication system 100 can be any type of system that transmits datapackets over a network. For example, the exemplary communication system100 can include one or more networks transmitting data packets acrosswired or wireless networks to terminals (terminals not shown in FIG. 1).The exemplary communication system 100 can have network architecturesof, for example, a GSM network, a UMTS network that adopts Wideband CodeDivision Multiple Access (W-CDMA) radio access technology, a CDMA2000network, and a WiMax network.

The exemplary communication system 100 can include, among other things,one or more networks 101, 102, 103(A-D), one or more controllers104(A-D), one or more serving nodes 105(A-B), one or more base stations106(A-D)-109(A-D), a router 110, a gateway 120, and one or more adaptivetraffic managers 130(A-C). At a high level, the network topology of theexemplary communication system 100 can have a tree-like topology withgateway 120 being the tree's root node and base stations 106-109 beingthe leaves.

Router 110 is a device that is capable of forwarding data packetsbetween computer networks, creating an overlay Internetwork. Router 110can be connected to two or more data lines from different networks. Whena data packet comes in on one of the lines, router 110 can determine theultimate destination of the data packet and direct the packet to thenext network on its journey. In other words, router 110 can perform“traffic directing” functions. In the exemplary embodiment shown in FIG.1, router 110 communicates with network 102 and gateway 120. Router 110directs traffic from the network 102 to the gateway 120 and vice versa.

Network 101 can be any combination of radio networks, wide area networks(WANs), local area networks (LANs), or wireless networks suitable forpacket-type communications, such as Internet communications. Forexample, in some exemplary embodiments, network 101 can be a GeneralPacket Radio Service (GPRS) core network, which provides mobilitymanagement, session management and transport for Internet Protocolpacket services in GSM and W-CDMA networks. The exemplary network 101can include, among other things, a gateway 120, and one or more servingnodes 105(A-B).

Gateway 120 is a device that converts formatted data provided in onetype of network to a particular format required for another type ofnetwork. Gateway 120, for example, may be a server, a router, a firewallserver, a host, or a proxy server. Gateway 120 has the ability totransform the signals received from router 110 into a signal thatnetwork 101 can understand and vice versa. Gateway 120 may be capable ofprocessing webpage, image, audio, video, and T.120 transmissions aloneor in any combination, and is capable of full duplex media translations.In some embodiments, gateway 120 can be a Gateway GPRS Support Node(GGSN) that supports interworking between the GPRS network and externalpacket switched networks, like the Internet and X.25 networks.

Serving nodes 105 are devices that deliver data packets from gateway 120to a corresponding network 103 within its geographical service area andvice versa. A serving node 105 can be a server, a router, a firewallserver, a host, or a proxy server. A serving node 105 can also havefunctions including packet routing and transfer, mobility management(attach/detach and location management), logical link management,network accessing mediation and authentication, and charging functions.As an exemplary embodiment, a serving node 105 can be a Serving GPRSSupport Node (SGSN). SGSN can have location register, which storeslocation information, e.g., current cell, current visitor location(Visitor Location Register) and user profiles, e.g., InternationalMobile Subscriber Identity (IMSI), and addresses used in the packet datanetwork, of all GPRS users registered with this SGSN.

Network 102 can include any combination of wide area networks (WANs),local area networks (LANs), or wireless networks suitable forpacket-type communications. In some exemplary embodiments, network 102can be, for example, Internet and X.25 networks. Network 102 cancommunicate data packet with network 101 with or without router 110.

Networks 103 can include any radio transceiver networks within a GSM orUMTS network or any other wireless networks suitable for packet-typecommunications. In some exemplary embodiments, depending on theunderlying transport technology being utilized, the Radio Access Network(RAN) or Backhaul area of network 103 can have a ring topology. In someembodiments, network 103 can be a RAN in a GSM system or a Backhaul areaof a UMTS system. The exemplary network 103 can include, among otherthings, base stations 106-109 (e.g., base transceiver stations (BTSs) orNode-Bs), and one or more controllers 104(A-C) (e.g., base-stationcontrollers (BSCs) or radio network controllers (RNCs)). Mobileterminals (not shown in FIG. 1) communicate with BTS/Node-B 106-109which have radio transceiver equipment. BTS/Node-B 106-109 communicatewith BSC/RNC 104(A-C), which are responsible for allocation of radiochannels and handoffs as users move from one cell to another. TheBSC/RNC 104(A-C) in turn communicate to serving nodes 105, which managemobility of users as well as provide other functions such as mediatingaccess to the network and charging.

As shown in FIG. 1, adaptive traffic manager 130 can be deployed at oneor more locations within communication system 100, including variouslocations within network 101 and 103. In some embodiments, adaptivetraffic manager 130 can be located at gateway 120, at controller 104, atone or more base stations 106-109, or any other locations. Adaptivetraffic manager 130 can be either a standalone network element or can beintegrated into existing network elements such as gateway 120,controllers 104, and base stations 106-109. Adaptive traffic manager 130can continuously monitor several parameters of communication system 100.The parameters can be used to generate traffic management rules. Thetraffic management rules are generated dynamically and change inreal-time based on the monitored parameters. After the rules aregenerated in real time, the rules are applied to data traffic beinghandled by adaptive traffic manager 130. Moreover, adaptive trafficmanager 130 can include a QoE parameter estimator 220 (shown in FIG. 2)for estimating QoE parameters in secured or unsecured transactions. QoEparameter estimator 220 is described in more detail below.

To optimize multimedia traffic, traffic management techniques can beimplemented on a proxy device (e.g., adaptive traffic manager 130) thatis located somewhere between a content server and client devices (e.g.,mobile terminals). The proxy device can determine the type of contentrequested by a specific mobile terminal (e.g., video content) and applyoptimization techniques. The content providers can transmit multimediacontent using unsecured or secured communication protocols such ashypertext transfer protocol secure (HTTPS) protocols, TLS protocols, andSSL protocols. The proxy device can determine the type of content beingtransmitted in both unsecured and secured transactions using clientrequests and server responses. In secured transactions, the clientrequests and server responses are encrypted and therefore may not bedecipherable by the proxy device.

Moreover, a variety of multimedia protocols above the HTTP layer areavailable for transmitting of multimedia contents. The protocols cangenerally be classified into two broad categories: progressive download(PD) protocols and adaptive bit rate (ABR) protocols. Examples ofadaptive bit-rate protocols include HTTP live streaming (HLS) protocols,dynamic adaptive streaming over HTTP (DASH) protocols, and smoothstreaming. Examples of PD protocols include flash video (FLV) file andMpeg-4 (MP4) file downloads over HTTP.

For both PD and adaptive bit-rate protocols, multiple quality levels(e.g., video quality level of 1080p, 720p, etc.) of the same multimediacontent can be stored at the server for transmitting to client devices.In the case of transmitting of multimedia content using PD protocols,the multimedia quality level requested by the client device cannot bechanged after the initial selection at the beginning of transmission. Inthe case of transmitting of multimedia content using adaptive bit-rateprotocols, the multimedia quality level requested by the client devicecan be changed to reflect fluctuations in the available networkbandwidth between the server and the client device. Therefore, adaptivebit-rate protocols typically provide a better user experience becausethe highest available quality level can be selected based on theavailable network bandwidth.

To apply traffic management techniques, such as to apply streamingpolicy control (SPC) to the transmission of multimedia contents, it isusually required to estimate QoE parameter associated with a specificterminal. Streaming policy control can be any traffic managementtechnique that manages data flow or controls congestion associated withstreaming of multimedia data across a network, such as the Internet. Forexample, SPC can allow streaming of the multimedia content to moreeffectively share bandwidths with other network traffics. SPC can alsoimproves smoothness in streaming and provide decreased and morepredictable latency.

FIG. 2 is a block diagram illustrating an exemplary adaptive trafficmanager 130, consistent with embodiments of the present disclosure. Insome embodiments, as shown in FIG. 2, QoE parameter estimator 220 can beintegrated with adaptive traffic manager 130. In some embodiments, QoEparameter estimator 220 can be integrated into other existing networkelements such as gateway 120, controllers 104, and/or one or more basestations 106-109. QoE parameter estimator 220 can also be a standalonenetwork element located at gateway 120, controller 104, one or more basestations 106-109, or any other proper locations.

As shown in FIG. 2, adaptive traffic manager 130 can include, amongother things, a traffic processing and policy enforcement unit 222 andQoE parameter estimator 220. Adaptive traffic manager 130 can have oneor more processors and at least one memory for storing programinstructions. The processor(s) can be a single or multiplemicroprocessors, field programmable gate arrays (FPGAs), or digitalsignal processors (DSPs) capable of executing particular sets ofinstructions. Computer-readable instructions can be stored on a tangibleand/or non-transitory computer-readable medium, such as random accessmemory (RAM), read-only memory (ROM), volatile memory, nonvolatilememory, hard drives, compact disc read-only memory (CD ROM), digitalversatile disc (DVD) memory, flash drives, magnetic strip storage,semiconductor storage, optical disc storage, magneto-optical discstorage, flash memory, registers, caches, and/or any other storagemedium. Singular terms, such as “memory” and “computer-readable storagemedium,” can additionally refer to multiple structures, such as aplurality of memories and/or computer-readable storage mediums.Alternatively, the methods can be implemented in hardware components orcombinations of hardware and software such as, for example, ASICs or oneor more computer.

Adaptive traffic manager 130 can obtain external data 201 forprocessing. External data 201 can include network probes, RemoteAuthentication Dial-In User Service (RADIUS), Policy Charging and RulesFunction (PCRF), and Subscriber Profile Repository (SPR). Adaptivetraffic manager 130 can also communicate with one or more terminals(e.g., client 210) and one or more servers (e.g., server 260), eitherdirectly or indirectly. Adaptive traffic manager 130 can include, forexample, a traffic processing and policy enforcement (TPPE) unit 222 anda QoE parameter estimator 220. Each of these components can be one ormore modules, which can be one or more packaged functional hardwareunits designed for use with other components or a part of a program thatperforms a particular function, corresponding to the particular step, ofrelated functions. QoE parameter estimator 220 can also include a datastorage 234, which can also be external to QoE parameter estimator 220.

In some embodiments, QoE parameter estimator 220 can include, amongother things, a video-audio transaction identifier 224, a quality levelvariation detector 226, a multimedia transaction processor 228, avideo-audio transaction aligner 230, a parameter estimator 232, and adata storage 234. In some embodiments, video-audio transactionidentifier 224 can identify transactions as audio transactions or videotransactions, as illustrated using FIGS. 4A-4B. Quality level variationdetector 226 can detect a quality level variation event using the videotransactions and/or the audio transactions, as illustrated in FIGS.5A-5B. Multimedia transaction processor 228 can process the videotransactions and/or audio transactions based on the detected qualitylevel variation event, as illustrated in FIGS. 6A-6B. Video-audiotransaction aligner 230 can map the processed video transactions andaudio transactions, as illustrated in FIGS. 7A-7C. Parameter estimator232 can estimate one or more QoE parameters using the processed videotransactions and audio transactions, as illustrated in FIG. 8. Theoperation of adaptive traffic manager 130 and its components are furtherdescribed using FIGS. 3-8 below.

FIG. 3 is a flowchart representing an exemplary method 300 forestimating QoE parameters associated with a specific terminal,consistent with embodiments of the present disclosure. Referring to FIG.3, it will be readily appreciated that the illustrated procedure can bealtered to delete steps or further include additional steps. Method 300can be performed by adaptive traffic manager 130, and more particularlyby QoE parameter estimator 220 of the adaptive traffic manager 130.While method 300 is described as being performed by QoE parameterestimator 220, it is appreciated that other components of adaptivetraffic manager 130 or other devices can be involved. Further, it isappreciated that any other adaptive traffic manager can also performmethod 300.

Referring to FIG. 3, after an initial step 310, TPPE unit 222 canacquire (step 320) a plurality of transactions for providing multimediacontent to specific terminal (e.g., client 210). In some embodiments, atleast one of the transactions is a secured transaction. For example, themultimedia content associated one or more transactions can be encrypted.TPPE unit 222 is a lower stack in the processing stack of adaptivetraffic manager 130. TPPE unit 222 can receive multimedia content, whichcan include video and/or web data, and provide the multimedia content toother elements and/or modules in adaptive traffic manager 130. Themultimedia content can be stored in a data storage device (e.g., datastorage 234) that is local to or remote from adaptive traffic manager130. TPPE unit 222 is responsible for routing traffic between client 210and the server 260. TPPE unit 222 can also implement traffic managementtechniques including blocking, rate limiting, lossless and lossy datacompression, and other traffic optimization techniques. TPPE unit 222can be a software program and/or a hardware device.

After TPPE unit 222 acquires the transactions, QoE parameter estimator220 can detect (step 330) a quality level variation event based on theplurality of transactions and the sizes of the plurality oftransactions; and estimate (step 340) the one or more QoE parametersbased on the detection of the quality level variation event. In someembodiments, the estimation of the one or more QoE parameters can beused for optimizing or reporting of the multimedia content provided tothe specific terminal. More details of steps 330 and 340, which can beperformed by various components of QoE parameter estimator 220, aredescribed using FIGS. 4A-8 below. After step 340, method 300 can proceedto a stop 350. It is appreciated that method 300 can also be repeatedfor detecting more quality level variation events and for estimatingmore QoE parameters.

Referring to FIG. 2, QoE parameter estimator 220 can include, amongother things, video-audio transaction identifier 224, quality levelvariation detector 226, multimedia transaction processor 228,video-audio transaction aligner 230, parameter estimator 232, and datastorage 234. While FIG. 2 illustrates that these components of QoEparameter estimator 220 are separate from each other, it is appreciatedthat one or more of these components can be integrated together to forman integral piece.

FIG. 4A is a flowchart representing an exemplary method 400 foridentifying video transactions and audio transactions, consistent withembodiments of the present disclosure. Referring to FIG. 4A, it will bereadily appreciated that the illustrated procedure can be altered todelete steps or further include additional steps. Method 400 can beperformed by adaptive traffic manager 130, and more particularly byvideo-audio transaction identifier 224 of the adaptive traffic manager130. While method 400 is described as being performed by video-audiotransaction identifier 224, it is appreciated that other components ofadaptive traffic manager 130 or other devices can be involved. Further,it is appreciated that any other adaptive traffic manager can alsoperform method 400.

Referring to FIG. 4A, video-audio transaction identifier 224 canidentify transactions as audio transactions or video transactions. Insome embodiments, after an initial step 410, video-audio transactionidentifier 224 can determine (step 415) whether more transactions areavailable for identification or whether all transactions have beenidentified. If all transactions have been identified as either videotransactions or audio transactions, video-audio transaction identifier224 proceeds to a stop 460. If more transactions are available foridentification, video-audio transaction identifier 224 can obtain (step420) a current transaction for identification. In some embodiments, eachtransaction can be identified corresponding to the order it wasacquired, or identified in any other desired order.

To identify the current transaction as an audio transaction or a videotransaction, video-audio transaction identifier 224 can obtain (step425) one or more neighboring transactions of the current transaction.Neighboring transactions are transactions that are communicated oracquired in proximity of time. For example, video-audio transactionidentifier 224 can obtain five neighboring transactions that are closestin time to the current transaction.

Neighboring transactions can include video and audio transactions thatare associated with the same media time segment. A media time segmentcan correspond to a time segment having a certain length of the mediatime. A media time length of multimedia content associated with a singlevideo or audio transaction can be, for example, five seconds. Thus, amedia time segment can have a length of, for example, five seconds,where the segment begins, for example, at 30 seconds and end at 35seconds. In some embodiments, if the current transaction is an audiotransaction, the neighboring transactions of the current transaction canhave a higher probability to include a video transaction thatcorresponds to the same media time segment of the audio transaction, andvice versa. For example, if the current transaction is an audiotransaction associated with a media time segment from 30 seconds to 35seconds, one of its neighboring transactions has a higher probability toinclude a video transaction that is associated with the same media timesegment, i.e., a media time segment from 30 seconds to 35 seconds.

Referring to FIG. 4A, video-audio transaction identifier 224 candetermine (step 430) a size of the current transaction and determine(step 435) an average size of the current transaction and the one ormore neighboring transactions. Video transactions can have sizes thatare different from those of audio transactions. For example, videotransactions can have sizes of 64-80 Kilo-Bytes (KB), 160-176 KB,320-336 KB, 384-400 KB, 704-720 KB, etc. Different sizes of the videotransactions correspond to different video quality levels. Audiotransactions can have sizes of, for example, 23-48 KB, 72-80 KB, etc. Insome embodiments, the number of different sizes of audio transactions(e.g., two) is less than the number of different sizes of videotransactions (e.g., five), reflecting that audio transactions can haveless number of quality levels than that of the video transactions.

After obtaining the sizes, video-audio transaction identifier 224 candetermine (step 440) a relation of the size of the current transactionand the average size of the current transaction and the one or moreneighboring transactions, such as determine whether the size of thecurrent transaction is less than or equal to the average size. In someembodiments, the quality level of a video transaction can correspond tothe quality level of an audio transaction. For example, if the videotransaction has a low video quality level, the audio transactionassociated with the same media time segment as that of the videotransaction can also have a low audio quality level. Therefore, the sizeof a video transaction can be greater than the size of the audiotransactions associated with the same media time segment. As describedabove, the neighboring transactions of the current transaction can havea high probability to include an audio transaction or a videotransaction associated with the same media time segment as that of thecurrent transaction. As a result, the average size of the currenttransaction and the neighboring transactions can be less than the sizeof a video transaction, but greater than the size of an audiotransaction.

An example of the relation between the sizes of audio transactions,video transactions, and average transactions are illustrated in FIG. 4B,which is a diagram illustrating exemplary relations 490 between averagetransaction sizes, the sizes of identified video transactions, and thesizes of identified audio transactions. Referring to FIG. 4B, curve 492illustrates a relation between transaction sizes of video transactionsand elapsed times, also referred to as wall-clock times. In someembodiments, the elapsed time represents the time at which a videotransaction or an audio transaction is completed. Curve 494 illustratesa relation between elapsed times and the moving average transactionsizes of the current transaction and neighboring transactions. Curve 496illustrates a relation between transaction sizes of audio transactionsand elapsed times. As shown in FIG. 4B, in some embodiments, the movingaverage transaction sizes (illustrated by curve 494) are greater thanthe audio transaction sizes (illustrated by curve 496), but are lessthan the video transaction sizes (illustrated by curve 492).

Referring back to FIG. 4A, after determining the relation of the size ofthe current transaction and the average size of the current transactionand the neighboring transactions, video-audio transaction identifier 224can identify the current transaction as an audio transaction or a videotransaction based on the determined relation. For example, if the sizeof the current transaction is greater than the average size of thecurrent transaction and the neighboring transactions, video-audiotransaction identifier 224 can identify (step 445) the currenttransaction as a video transaction. If the size of the currenttransaction is less than or equal to the average size of the currenttransaction and the neighboring transactions, video-audio transactionidentifier 224 can identify (step 450) the current transaction as anaudio transaction. After the current transaction is identified,video-audio transaction identifier 224 can move to identify the nexttransaction until all transactions are identified as audio transactionsor video transactions. Method 400 can then proceed to a stop 460.

FIG. 5A is a flowchart representing an exemplary method 500 fordetecting a quality level variation event, consistent with embodimentsof the present disclosure. Referring to FIG. 5A, it will be readilyappreciated that the illustrated procedure can be altered to deletesteps or further include additional steps. Method 500 can be performedby adaptive traffic manager 130, and more particularly by quality levelvariation detector 226 of the adaptive traffic manager 130. While method500 is described as being performed by quality level variation detector226, it is appreciated that other components of adaptive traffic manager130 or other devices can be involved. Further, it is appreciated thatany other adaptive traffic manager can also perform method 500.

Referring to FIG. 2, quality level variation detector 226 can detect aquality level variation event using the video transactions and/or audiotransactions. The quality level for transmitting multimedia contentbetween server 260 and client 210 can vary due to many reasons. Forexample, under certain protocols such as adaptive bitrate (adaptivebit-rate) protocols, during the beginning of the transmission (e.g., afew seconds), client 210 can estimate the channel bandwidth anddynamically change the quality level of the multimedia content beingtransmitted. As an example, if the current quality level is low (e.g., avideo quality level of 240p) and client 210 determines that theavailable bandwidth is enough to increase the quality level by one ormore resolution steps, client 210 can request to increase the qualitylevel (e.g., to a video quality level of 720p) in subsequent multimediacontent it receives from server 260.

As another example, if client 210 determines that the current availablebandwidth is not enough to support the current quality level (e.g., avideo quality level of 1080p) associated with the multimedia content,client 210 can request to decrease the quality level (e.g., to a videoquality level of 720p) in subsequent multimedia content it receives fromserver 260. As a result, the quality level can vary from time to timedue to bandwidth availability or any other reasons. A quality level canbe a video quality level or an audio quality level. While the aboveexample uses video quality level, it is appreciated that audio qualitylevel can also vary from time to time.

In some embodiments, when a quality level varies during the transmittingof the multimedia content, one or more transactions can include the samemultimedia content at different quality levels. For example, before aquality level varies from a first quality level (e.g., low qualitylevel) to a second quality level (e.g., high quality level) at a certainelapsed time (e.g., 35 seconds), multimedia content associated with oneor more media time segments (e.g., 20-25 seconds, 25-30 seconds, 30-35seconds, etc.) at the first quality level can have been alreadycommunicated in a plurality of transactions between client 210 andserver 260. The plurality of transactions can be acquired by adaptivetraffic manager 130.

After the quality level varies, the same multimedia content associatedwith the same media time segments (e.g., 20-25 seconds, 25-30 seconds,30-35 seconds, etc.) at the second quality level can be communicated ina plurality of additional transactions between client 210 and server260. The additional transactions can also be acquired by adaptivetraffic manager 130. As a result, transactions including the samemultimedia content associated with the same media time segments at twodifferent quality levels can both be acquired by adaptive trafficmanager 130. In some embodiments, multimedia transaction processor 228of adaptive traffic manager 130 can discard the transactions includingthe same multimedia content associated with same media time segments atthe first quality level. Multimedia transaction processor 228 aredescribed in more detail below.

The detection of the quality level variation event can have an impact onthe estimation of QoE parameters such as video bitrate. For example,estimating the video bitrate can be based on the transaction sizes ofthe transactions that are not discarded.

When the quality level variation event occurs, the sizes of thetransactions can vary. For example, after a video quality levelincreases (e.g., from a quality level of 720p to 1080p), the transactionsize can increase (e.g., from 384-400 KB to 704-720 KB). Under certainprotocols such as the adaptive bit-rate protocols, the variation of thetransaction sizes can occur even if the quality level does not vary. Forexample, while a first transaction and a second transaction can eachinclude multimedia content associated with a five-second media timesegment at the same quality level, the multimedia content included inthe first transaction can have mostly still images, while multimediacontent included in the second transaction can have frequently varyingimages (e.g., images depicting actions). As a result, the transactionsize of the second transaction can be greater than that of the firsttransaction, although the two transactions both have the same media timesegments at the same quality level.

Referring to FIG. 5A, quality level variation detector 226 can performmethod 500 to detect quality level variation events with respect tovideo transactions or audio transactions. Using video transactions as anexample, quality level variation detector 226 can determine (step 515)whether more video transactions are available for quality levelvariation detection. If no more video transactions are available,quality level variation detector 226 can proceed to a stop 560. If moretransactions are available, quality level variation detector 226 canobtain (step 520) a current video transaction for quality leveldetection and obtain (step 525) one or more preceding video transactionsand one or more following video transactions. The preceding videotransactions are transactions that were communicated or were acquiredbefore the current video transaction. And the following videotransactions are transactions that were communicated or were acquiredafter the current video transaction. In some embodiments, quality levelvariation detector 226 can configure the number (e.g., two) of thepreceding video transactions and the number of following videotransactions to be obtained. Based on the configuration, for example,two preceding video transactions and two following video transactionscan be obtained. It is appreciated that any other number of thepreceding video transactions and the following video transactions canalso be obtained for detection of quality level variation.

Referring to FIG. 5A, after obtaining the current video transaction, thepreceding video transactions, and the following video transactions,quality level variation detector 226 can determine (step 530) the sizeof the current video transaction, and determine (step 535) the averagesize of the one or more preceding video transactions and the averagesize of the one or more following video transactions. The average sizeis the total size of the transactions divided by the number of thetransactions. For example, the average size of the preceding videotransactions or following video transactions can be the total size oftwo preceding video transactions or two following video transactions,respectively, divided by two.

After the sizes are determined, quality level variation detector 226 canobtain (step 540) a first video size difference and a second video sizedifference. The first video size difference can be the differencebetween the size of the current video transaction and the average sizeof the preceding video transactions. The second video size differencecan be the difference between the size of the current video transactionand the average size of the following video transactions. Using at leastone of the first video size difference and the second video sizedifference, quality level variation detector 226 can detect the qualitylevel variation event.

As an example, quality level variation detector 226 can determine (step545) whether the first video size difference satisfies a first videosize difference threshold and whether the second video size differencesatisfies a second video size difference threshold. If both aresatisfied, quality level variation detector 226 can determine (step 550)that a quality level variation event occurs. For example, the firstvideo size difference threshold can be 40% and the second video sizedifference threshold can be 5%. If quality level variation detector 226determines that the first video size difference (e.g., the absolutevalue of the size difference between the current video transaction andthe one or more preceding video transactions) is greater than or equalto 40%, and that the second video size difference (e.g., the absolutevalue of the size difference between the current video transaction andthe one or more following video transactions) is less than 5%, qualitylevel variation detector 226 can determine that a quality levelvariation event occurs. It is appreciated that the first video sizedifference threshold and the second video size difference threshold canbe any other values (e.g., 50%, 1%, etc.).

Referring to FIG. 5A, in some embodiments, if quality level variationdetector 226 determines (step 545) that either the first video sizedifference does not satisfy the first video size difference threshold orthe second video size difference does not satisfy the second video sizedifference threshold, quality level variation detector 226 can determinethat no quality level variation event occurs and the method proceeds tostep 515.

As mentioned above, it is appreciated that quality level variationdetector 226 can also perform method 500 with respect to audiotransactions. The details are similar to those described above withrespect to video transactions. It is appreciated that method 500 canalso be repeated to detect more quality level variation events.

FIG. 5B is a diagram illustrating exemplary relations 590 of transactionsizes variation and the detection of quality level variation, consistentwith embodiments of the present disclosure. Referring to FIG. 5B, curve592 illustrates a relation between transaction sizes of video or audiotransactions and elapsed times, also referred to as wall-clock times.Curve 594 illustrates a relation between a quality level detectionindicator and the elapsed times. As shown in FIG. 5B, in someembodiments, when a video or audio transaction (e.g., the transaction atelapsed time of about 19 seconds) has a size that is substantiallydifferent (e.g., different by 40%) from the preceding transactions andis substantially similar (e.g., within 5%) to the followingtransactions, the quality level detection indicator changes from, forexample, “0” to “1”.

In some embodiments, as shown in FIG. 5B, when obtaining one or morepreceding video transactions or audio transactions, quality levelvariation detector 226 can determine that one or more of such precedingtransactions are spurious (e.g., the transaction at elapsed time 10-12seconds) and ignore those transactions. Spurious transactions reflect atemporary or momentary change of the transaction size due to a momentaryquality level change or other short duration events.

FIG. 6A is a flowchart representing an exemplary method 600 forprocessing video transactions or audio transactions based on thedetected quality level variation event, consistent with embodiments ofthe present disclosure. Referring to FIG. 6A, it will be readilyappreciated that the illustrated procedure can be altered to deletesteps or further include additional steps. Method 600 can be performedby adaptive traffic manager 130, and more particularly by multimediatransaction processor 228 of the adaptive traffic manager 130. Whilemethod 600 is described as being performed by multimedia transactionprocessor 228, it is appreciated that other components of adaptivetraffic manager 130 or other devices can be involved. Further, it isappreciated that any other adaptive traffic manager can also performmethod 600.

As described above, quality level variation detector 226 can detect aquality level variation event based on the sizes of the video or audiotransactions. Referring to FIG. 6A, after an initial step 610,multimedia transaction processor 228 can obtain (step 620) an elapsedtime corresponding to the quality level variation event. As discussedabove, the quality level variation event can indicate a variation ofquality level from a first quality level to a second quality level. Forexample, the quality level variation event can indicate a video qualitylevel variation from 720p to 1080p. After such quality level variationevent is detected, multimedia transaction processor 228 can obtain theelapsed time (e.g., 19 seconds) that such quality level variation eventoccurred.

Referring to FIG. 6A, based on the elapsed time corresponding to thequality level variation event, multimedia transaction processor 228 canidentify (step 630) a specific transaction using the video transactionsand/or the audio transactions, where the multimedia content associatedwith the specific transaction has an actual cumulative media time thatis greater than or equal to the elapsed time. As an example, FIG. 6B isan exemplary timing table 660 illustrating the relations between elapsedtimes, the measured cumulative media time, and the actual cumulativemedia time. Referring to FIG. 6B, column 662 represents the indices oftransactions (e.g., index 0-13). For example, the index number 0transaction represents the first transaction in the plurality oftransactions. The index number 1 transaction represents the secondtransaction in the plurality of transactions, and so forth.

As shown in FIG. 6B, column 664 represents the elapsed timecorresponding to each of the transactions. The elapsed time, alsoreferred to as the wall-clock time, represents the time at which thetransaction is completed or acquired. For example, the seventhtransaction (i.e., the index number 6 transaction) has an elapsed timeof 19 seconds, which indicates that the transaction is completed oracquired at 19 seconds from the time the first transaction is completedor acquired (e.g., a transaction completed at 0 second corresponding tothe index number 0).

Referring to FIG. 6B, column 666 represents the measured cumulativemedia time corresponding to each of the transactions. As describedabove, each transaction can include multimedia content associated with amedia time segment having a certain media time length (e.g., 5 seconds).In some embodiments, a certain transaction can provide multimediacontent associated with a media time segment corresponding to a futuretime segment with respect to the elapsed time. For example, the indexnumber 0 transaction completes at an elapsed time of 0 second andprovides multimedia content associated with a media time segment of 0-5seconds. The index number 1 transaction completes at an elapsed time of4 seconds and provides multimedia content associated a media timesegment of 5-10 seconds, and so forth. The measured cumulative mediatime represents the ending time of the media time segment associatedwith the multimedia content included in a transaction when, for example,there is no quality level variation event. Thus, when there is noquality level variation, the measured cumulative media time for theindex number 1 transaction is 10 seconds.

Providing the multimedia content associated with media time segments ina future time allows client 210 to play the multimedia content withoutstalling. For example, as shown in FIG. 6B, when the elapsed time is at4 seconds (e.g., the index number 1 transaction), the measured or actualcumulative media time is 10 seconds, indicating that the multimediacontent associated with media time segments of 0-10 have been providedto client 210 (e.g., buffered in client 210). As a result, at elapsedtime of 4 seconds, client 210 can have 10 seconds of buffered multimediacontent and therefore stalling can be prevented.

In some embodiments, when a quality level variation event occurs at acertain elapsed time, one or more transactions having multimedia contentassociated with measured cumulative media times that are greater thanthe elapsed time may have already completed. Such transactions caninclude multimedia content at a quality level used prior to the qualitylevel variation event occurs. In some embodiments, after the qualitylevel variation event occurs, transactions including the same multimediacontents associated with the same multimedia time segment at a differentquality level can be communicated or acquired. As a result, thetransactions including multimedia content at the prior quality level canbe discarded.

For example, referring to FIG. 6B, a quality level variation eventoccurs at an elapsed time of 19 seconds. When such quality levelvariation event occurs, the quality level can vary from a first qualitylevel (e.g., a quality level of 760p) to a second quality level (e.g., aquality level of 1080p). The transaction at the elapsed time of 19seconds corresponds to index number 6 and the corresponding measuredcumulative media time is 35 seconds, indicating the media time segmentof index number 6 transaction is 30-35 seconds, if the media time lengthper segment is five seconds.

As shown in FIG. 6B, column 668 represents the actual cumulative mediatime. The actual cumulative media time represents the ending time of themedia time segment associated with the multimedia content included in atransaction when, for example, there is quality level variation event.After the quality level variation event occurs, the first transactionincluding multimedia content at the second quality level can providemultimedia content having media segment time starting at a time that isgreater than the elapsed time when the quality level variation eventoccurs. For example, the first transaction including multimedia contentat the second quality level can have a media segment starting at thenearest round-up time based on the media time length of the multimediacontent included in a single transaction. In the above example where thequality level variation event occurs at 19 seconds of elapsed time, ifthe media time length of the multimedia content included in a singletransaction is five seconds, the first transaction including themultimedia content at the second quality level can have a media timesegment starting at 20 seconds (i.e., the nearest round-up time of 19seconds using five seconds of media time length per segment) and endingat 25 seconds.

Referring to FIG. 6B, in the above example, the index number 6transaction corresponds to a measured accumulative media time of 35seconds, which is greater than the actual cumulative media time of 25seconds (i.e., the ending time of the media time segment of the firsttransaction including the multimedia content at the second qualitylevel). This indicates that one or more transactions having elapsedtimes that are less than the time when the quality variation eventoccurs may include multimedia content at the first quality level (e.g.,a quality level of 760p). Such transactions may not be desired becausethe same multimedia content at the second quality level (e.g., a qualitylevel of 1080p) are included in the transactions having elapsed timethat is greater than the time when quality level variation event occurs.

In the above example where the quality level variation event occurs at19 seconds of elapsed time, the first transaction including themultimedia content at the second quality level has a media time segmentstarting at 20 seconds. As shown in FIG. 6B, for example, the indexnumber 4 and 5 transactions have measured cumulative media times of 25seconds and 30 seconds, respectively. The measured cumulative mediatimes of 25 seconds and 30 seconds indicate that their correspondingmedia time segments begin at 20 seconds and 25 seconds, respectively, ifthe media time length is five seconds. As a result, the index number 4and 5 transactions include multimedia content at the first qualitylevel, where the same multimedia content at the second quality level arecommunicated or acquired. Thus, these transactions are not required andcan be discarded.

As shown in FIG. 6A, for discarding these duplicate transactions,multimedia transaction processor 228 can identify (step 630) a specifictransaction. The multimedia content associated with the specifictransaction has a measured cumulative media time that is greater than orequal to a round-up time based on the elapsed time at which the qualitylevel variation event occurs. In the example from FIG. 6B, the specifictransaction can be the index number 3 transaction, which corresponds toa measured cumulative media time of 20 seconds. If the media time lengthper segment is five seconds, the round-up time of the elapsed time atwhich the quality level variation event occurs (i.e., 19 seconds) is 20seconds. Thus, the index number 3 transaction has a measured cumulativemedia time that equals the round-up time based on the elapsed time atwhich the quality level variation event occurs. As a result, the indexnumber 3 transaction can be identified as the specific transaction.

Referring back to FIG. 6A, after identifying the specific transaction,multimedia transaction processor 228 can discard (step 640) one or moretransactions following the specific transaction. The one or morediscarded transactions include multimedia content at the first qualitylevel, i.e., the quality level used prior to the quality level variationevent occurs. Referring to FIG. 6B, in the above example where thequality level variation event occurs at 19 seconds of elapsed time, theindex number 4 and 5 transactions include multimedia content at thefirst quality level and are thus discarded.

Column 668 illustrates the actual cumulative media time used by a client(e.g., client 210) after multimedia transaction processor 228 discardsthe index number 4 and 5 transactions that include multimedia content atthe first quality level. As shown in column 668 of FIG. 6B, the actualcumulative media time used by client 210 remains at 20 seconds at indexnumbers 4 and 5. At index number 6, the quality level variation eventoccurs and the quality level is changed from the first quality level toa second quality level. Transactions that include multimedia content atthe second quality level can be communicated or acquired. For example,at index number 6, a transaction can include multimedia contentassociate with a media time segment at the second quality level, wherethe media time segment is 20-25 seconds. Therefore, actual cumulativemedia time at the index number 6 transaction is 25 seconds. Thesubsequent transactions (e.g., the index number 7-13 transactions) caninclude multimedia content at the second quality level and thus theactual cumulative media time increases as illustrated in column 668.

Referring back to FIG. 6A, after discarding one or more transactionincluding multimedia content at the first quality level, method 600 canproceed to a stop 650. It is appreciated that method 600 for processingtransactions can be performed to process video transactions, audiotransactions, or both. After such processing, including discarding oneor more video and/or audio transactions due to the quality levelvariation events, the number of the remaining video transactions can bedifferent from the number of the remaining audio transactions.Subsequently, a mapping between the remaining video transactions and theremaining audio transactions can be performed. In some embodiments, themapping enables aligning of the video content included in the remainingvideo transactions and the audio content included in the audiotransactions, such that client 210 can play the video and audio contentin a synchronized manner.

FIG. 7A is a flowchart representing an exemplary method 700 foridentifying a mapping between video transactions and audio transactions,consistent with embodiments of the present disclosure. Referring to FIG.7A, it will be readily appreciated by one of ordinary skill in the artthat the illustrated procedure can be altered to delete steps or furtherinclude additional steps. Method 700 can be performed by adaptivetraffic manager 130, and more particularly by video-audio transactionaligner 230 of the adaptive traffic manager 130. While method 700 isdescribed as being performed by video-audio transaction aligner 230, itis appreciated that other components of adaptive traffic manager 130 orother devices can be involved. Further, it is appreciated that any otheradaptive traffic manager can also perform method 700.

Referring to FIG. 7A, video-audio transaction aligner 230 can obtain(step 720) video transaction timestamps associated with the remainingvideo transactions and obtain (step 730) audio transaction timestampsassociated with the remaining audio transactions. A timestamp associatedwith a video or an audio transaction represents the time that thetransaction is communicated or acquired. A timestamp can be an elapsedtime measured relative to the first transaction for transmittingmultimedia content. In some embodiments, the timestamp of the firsttransaction can be set to time “0”. A subsequent transaction can have atimestamp that is equal to the elapsed time measure from time “0”.

As an example, FIG. 7B is an exemplary table 780 illustrating exemplaryrelations between transaction indices, timestamps of the remaining audiotransactions, and timestamps of the remaining video transactions. Asshown in FIG. 7B, column 782 represents the transaction indices (e.g.,0-16) of the plurality of video or audio transactions. Column 784represents the audio transaction timestamps. In this example, there is atotal of seventeen remaining audio transactions having transactionindices of 0-16. The index number 0 audio transaction has a timestamp of“0”. The index number 1 audio transaction has a timestamp of “3”, soforth. Column 786 represents the video transaction timestamps. In thisexample, there is a total of twelve remaining video transactions havingtransaction indices of 0-11. The index number 0 video transaction has atimestamp of “0”. The index number 1 video transaction has a timestampof “4”, and so forth. As shown in FIG. 7B, the number of remaining audiotransactions can be different from the number of remaining videotransactions. And the timestamps of a specific audio transaction can bedifferent from a specific video transaction that has the sametransaction index. For example, for the same transaction index number 1,the audio transaction has a timestamp of “3” and the video transactionhas a timestamp of “4”, indicating that the audio transaction or thevideo transaction are not communicated, acquired, or completed at thesame time.

Referring back to FIG. 7A, after obtaining the timestamps associatedwith the remaining audio transactions and the remaining videotransactions, video-audio transaction aligner 230 can determine (step740) a score matrix based on the video transaction timestamps and theaudio transaction timestamps. In some embodiments, the score matrix canbe determined using a dynamic time warping (DTW) algorithm. The scoresin the score matrix can be the optimal or the least scores for matchinga video transaction and an audio transaction. For example, using a DTWalgorithm, video-audio transaction aligner 230 can determine a scorematrix S with elements S(i, j), where S(i, j) is the optimal or theleast score for matching the first “i” audio transactions and the first“j” video transactions. The number of rows and columns in the scorematrix S equals the number of the audio transactions and the number ofvideo transactions, respectively. As an example, video-audio transactionaligner 230 can use the following formulas to determine the elements ofthe score matrix S:

$\begin{matrix}{{S( {0,0} )} = {{abs}( {{a(0)} - {v(0)}} )}} & (1) \\{{{S( {0,j} )} = {{S( {0,{j - 1}} )} + {{abs}( {{a(0)} - {v(j)}} )}}},{{{for}\mspace{14mu} j} > 0}} & (2) \\{{{S( {i,0} )} = {{S( {{i - 1},0} )} + {{abs}( {{a(i)} - {v(0)}} )}}},{{{for}\mspace{14mu} i} > 0}} & (3) \\{{{S( {i,j} )} = {{{{abs}( {{a(i)} - {v(j)}} )} + {\min\begin{Bmatrix}{S( {{i - 1},{j - 1}} )} \\{S( {{i - 1},j} )} \\{S( {i,{j - 1}} )}\end{Bmatrix}\mspace{14mu}{for}\mspace{14mu} i}} > 0}},{j > 0}} & (4)\end{matrix}$

In the above formulas (1)-(4), a(i) represents the transaction timestampof the i^(th) audio transaction, and v(j) represents the transactiontimestamp of the j^(th) video transaction. Formulas (1)-(4) can beiteratively applied for determining the score matrix S.

FIG. 7C is an exemplary score matrix 790 illustrating an exemplarymapping between audio transactions and video transactions. For example,video-audio transaction aligner 230 can obtain score matrix 790 byapplying the above formulas (1)-(4) to the video and audio transactionsin table 780 illustrated in FIG. 7B.

Referring to FIG. 7C, the score of each matrix element S(i,j) of scorematrix 790 corresponds to the optimal score for matching the first iaudio transactions (e.g., the audio transactions having the audiotransaction timestamps listed in column 784 of table 780 and reproducedin column 791 of score matrix 790, i.e., the left most column) and thefirst j video transactions (e.g., the video transactions having thevideo transaction timestamps listed in column 786 of table 780 andreproduced in row 792 of score matrix 790, i.e., the top most row).

In some embodiments, the score of the matrix element corresponding tothe last row and the last column of score matrix S can represent thefinal optimal score for matching all the audio transactions and thevideo transactions. In the exemplary score matrix 790, the final optimalscore of 16 is illustrated in matrix element 794Q. Using a score matrixS, video-audio transaction aligner 230 can also obtain an optimalmatching path for all the video transactions and the audio transactions.Referring to FIG. 7C, for example, the optimal matching path includesmatrix elements 794A-Q.

Referring back to FIG. 7A, using the optimal matching path in the scorematrix, video-audio transaction aligner 230 can map (step 750) each ofthe remaining one or more video transactions to a correspondingremaining one or more audio transactions. The optimal matching pathincludes matrix elements that have the optimal or the least score in thecorresponding columns and/or rows of the matrix elements. For example,referring to FIG. 7C, matrix element 794A has a score of 0, which is theleast score in the matrix row containing matrix element 794A and alsothe least score in the matrix column containing matrix element 794A.Matrix element 794A corresponds to the video transaction having thevideo transaction timestamp of 0 second (i.e., the index number 0 videotransaction illustrated in column 786 of FIG. 7B) and also correspondsto the audio transaction having the audio transaction timestamp of 0second (i.e., the index number 0 audio transaction illustrated in column784 of FIG. 7B). Thus, based on the score of matrix element 794A,video-audio transaction aligner 230 can map the index number 0 videotransaction to the index number 0 audio transaction.

Similarly, matrix element 794B has a score of 1, which is the leastscore in the matrix row containing matrix element 794B and also theleast score in the matrix column containing matrix element 794B. Matrixelement 794B corresponds to the video transaction having the videotransaction timestamp of 4 seconds (i.e., the index number 1 videotransaction illustrated in column 786 of FIG. 7B) and the audiotransaction having the audio transaction timestamp of 3 seconds (i.e.,the index number 1 audio transaction illustrated in column 784 of FIG.7B). Thus, based on the score of matrix element 794A, video-audiotransaction aligner 230 can map the index number 1 video transaction tothe index number 1 audio transaction illustrated in table 780 of FIG.7B.

Video-audio transaction aligner 230 can also map other videotransactions and audio transactions based on scores of the matrixelements contained in the optimal matching path (e.g., matrix elements794C-Q of FIG. 7C). In some embodiments, the matrix elements containedin the optimal matching path are not required to be the optimal or leastscore in both the row and column it belongs to. For example, matrixelement 794Q has a score of 16, which is the least score in the columncontaining matrix element 794Q, but is not the least score in the rowcontaining matrix element 794Q (e.g., the least score is 14).

Referring to FIG. 7C, in some embodiments, video-audio transactionaligner 230 can map one or more audio transactions to a single videotransaction, or vice versa. For example, in score matrix 790, scorematrix elements 794F and 794G correspond to the audio transactionshaving timestamps of 21 and 22 seconds (i.e., the index number 5 and 6audio transactions illustrated in column 784 of FIG. 7B), respectively.Score matrix elements 794F has a score of 3, which is the least scoreamong those of the matrix row and the matrix column containing matrixelements 794F. Matrix elements 794G has a score of 4, which is the leastscore among those of the matrix row containing matrix elements 794G, andthe second least score among those of the matrix column containingmatrix elements 794G. Based on the scores of matrix elements 794F and794G, video-audio transaction aligner 230 can map these two audiotransactions to the video transaction having timestamps of 21 second(i.e., the index number 5 video transactions illustrated in column 786of FIG. 7B).

Similarly, while score matrix 790 illustrates the mapping two or moreaudio transactions to a single video transaction, it is appreciated thatvideo-audio transaction aligner 230 can also map two or more videotransactions to a single audio transaction when there are more videotransactions than audio transactions. It is further appreciated thatvideo-audio transaction aligner 230 can map audio transactions and videotransactions based on the any desired selection of the optimal matchingpath of a score matrix.

In some embodiments, after video-audio transaction aligner 230 maps twoor more transactions of one type to a single transaction of the othertype, video-audio transaction aligner 230 can retain one of the multipletransactions and discard the other transactions. Video-audio transactionaligner 230 can select the transaction for retaining based ondetermination of an optimal score among the scores of the matrixelements of corresponding to the multiple transactions.

For example, as shown in FIG. 7C, video-audio transaction aligner 230maps audio transactions having timestamps of 21 and 22 seconds (i.e.,the index number 5 and 6 audio transactions illustrated in column 784 ofFIG. 7B) to the video transaction having timestamps of 21 second (i.e.,the index number 5 video transactions illustrated in column 786 of FIG.7B). After such mapping, video-audio transaction aligner 230 can retainthe audio transactions having timestamps of 21 second (i.e., the indexnumber 5 audio transaction illustrated in column 784 of FIG. 7B) basedon the determination that optimal score among the scores of matrixelements 794F and 794G, i.e., 3 and 4, respectively, is 3.

As shown in FIG. 7A, after video-audio transaction aligner 230 maps theaudio transactions and the video transactions and discards one or moretransactions such that each of the audio transaction is mapped to adifferent video transaction, method 700 can proceed to a stop 760. It isappreciated that method 700 can also be repeated to map more audiotransactions and more video transactions.

FIG. 8 is a flowchart representing an exemplary method 800 fordetermining the one or more quality of experience (QoE) parameters basedon the mapped video transactions and audio transactions, consistent withthe embodiments of the present disclosure. Referring to FIG. 8, it willbe readily appreciated by one of ordinary skill in the art that theillustrated procedure can be altered to delete steps or further includeadditional steps. Method 800 can be performed by an adaptive trafficmanager 130, and more particularly by a parameter estimator 232 ofadaptive traffic manager 130. While method 800 is described as beingperformed by parameter estimator 232, it is appreciated that othercomponents of adaptive traffic manager or other devices can be involved.

Referring to FIG. 8, after an initial step 810, parameter estimator 232can determine (step 820) video bitrates and audio bitrates using theprocessed (e.g., mapped) audio and video transactions. In someembodiments, each transaction can include multimedia content having afixed media time length (e.g., 5 seconds) or a varied media time length.The audio bitrate or the video bitrate can be estimated based on themedia time length and the size of the transaction. In some embodiments,a transaction can include headers (e.g., HTTP headers). As a result, thesize of the multimedia content included in each transaction equals thedifference between the size of the transaction and the size of theheaders. When the transaction is an unsecured transaction, parameterestimator 232 can obtain the size of the headers. When the transactionis a secured transaction such as an encrypted transaction, parameterestimator 232 can estimate the approximate size of the header of thesecured transaction to have, for example, a fixed size (e.g., 500bytes).

In some embodiments, the audio bitrate equals the size of the audiocontent (i.e., the difference of the size of the audio transaction andthe size of the audio transaction headers) divided by the media timelength of the audio content included in the audio transaction.Similarly, the video bitrate equals size of the video content (i.e., thedifference of the size of the video transaction and the size of thevideo transaction headers) divided by the media time length of the videocontent included in the video transaction.

In some embodiments, parameter estimator 232 can estimate the sizes ofthe multimedia content included in each of the processed videotransactions and each of the processed audio transactions. Based onestimated sizes of the processed video transactions and the processedaudio transactions, parameter estimator 232 can estimate the averagevideo bitrate (e.g., 178 kbps) and the average audio bitrate (e.g., 74kbps), respectively. Further, using the estimated sizes of the processedvideo transactions and the processed audio transactions, parameterestimator 232 can also estimate variations of video bitrates and audiobitrates with respect to the average video bitrate and the average audiobitrate, respectively.

Referring to FIG. 8, parameter estimator 232 can determine (step 830) atotal media time associated with the multimedia content provided to aspecific terminal. The total media time equals the number oftransactions multiplied by the media time length per segment. Using theexample illustrated in FIGS. 7B and 7C, after the processing (e.g.,aligning) of the transactions, there are total of 12 video transactionsand 12 corresponding audio transactions. Each of the video or audiotransactions can include multimedia content having a media time lengthof 5 seconds. Accordingly, the total media time is 60 seconds (i.e., 12transactions×5 seconds).

Referring back to FIG. 8, parameter estimator 232 can further determine(step 840) a stalling event. In some embodiments, the total media timecan represent the amount of media time that is played by client 210.Parameter estimator 232 can detect stalling events based on the totalmedia time and one or more timestamps of the transactions. For example,parameter estimator 232 can estimate a total elapsed time using thetimestamp of the last transaction relative to timestamps of the firsttransaction when a multimedia session begins. A multimedia sessionincludes one or more audio transactions and video transactions.Parameter estimator 232 can obtain the difference between the totalelapsed time and the total media time, and determine that a stallingevent occurs if the total elapsed time is greater than the total mediatime.

Referring to FIG. 8, parameter estimator 232 can determine (step 850) asize of wasted data. In some embodiments, the adaptive bit-rateprotocols such as the HLS protocols can adjust the quality level basedon, for example, the available bandwidth for a specific terminal or thenetwork (e.g., network 101). Adjusting the quality levels can cause anundesirable side effect of wasting data. As described above, when aquality level variation occurs, a number of transactions havingmultimedia content at the old quality level can be discarded and thuswasted. As another example, under degraded network conditions or whenserver 260 introduces large delays while providing multimedia content,client 210 can repeatedly send a request for particular multimediacontent having certain media time segments that are previouslyrequested. In this case, one or more server responses can includeduplicate multimedia content, which are transmitted to client 210multiple times. In some cases, only one of these responses reachesclient 210 within the time required for a real-time playback of themultimedia content. The duplicated multimedia content included in theother server responses are thus wasted.

In some embodiments, parameter estimator 232 can determine the size ofthe wasted data during a multimedia session based on the size of theprocessed transactions and the size of the unprocessed transactions. Asdescribed above, after a quality level variation event occurs,multimedia transaction processor 228 can discard one or more audioand/or video transactions that include multimedia content associatedwith a quality level used prior to the quality level variation event.Further, video-audio transaction aligner 230 can discard one or moreaudio and/or video transactions such that the number of audiotransactions equals the number of video transactions. Parameterestimator 232 can estimate the size of these discarded transactions.Parameter estimator 232 can thus estimate the size of wasted data basedon the size of these discarded transactions. For example, if the totalsize of all video transactions is 1,906,192 bytes and the size of theprocessed video transactions (i.e., excluding discarded videotransactions) is 1,333,219 bytes, the percentage of the wasted videodata is thus 30%, that is, the difference of the size of the unprocessedvideo transactions and processed video transactions divided by the sizeof the unprocessed video transactions. Similarly, if the total size ofall audio transactions is 1,027,644 bytes and the size of the processedaudio transactions (i.e., excluding the discarded audio transactions) is553,562 bytes, the percentage of the wasted audio data is thus 46%. Theoverall wasted data, including the wasted audio data and the wastedvideo data, is thus 36%.

In some embodiments, the wasted data can be represented using the numberof discarded transactions. For example, if the total number of videotransactions is 14 and the number of the processed video transactions is12, the percentage of the wasted video data is 14%. Similarly, if thetotal number of audio transactions is 21 and the number of the processedaudio transactions is 12, the percentage of the wasted audio data isthus 43%. The overall wasted data, including the wasted audio data andthe wasted video data, is thus 31%.

In some embodiments, the wasted data can be represented using the lengthof media time segments. Similar to the above examples, the wasted datacan be represented in terms of wasted video data using the length ofmedia time segments in the discarded video transactions, in terms ofwasted audio data using the length of media time segments in thediscarded audio transactions, and in terms of an overall waste data.Referring to FIG. 8, after step 850, method 800 can proceed to a stop860. Method 800 can also be repeated any desired number of times forestimating the parameters.

In the foregoing specification, an element (e.g., adaptive trafficmanager) can have one or more processors and at least one memory forstoring program instructions corresponding to methods 300, 400, 500,600, and 700, consistent with embodiments of the present disclosure. Theprocessor(s) can be a single or multiple microprocessors, fieldprogrammable gate arrays (FPGAs), or digital signal processors (DSPs)capable of executing particular sets of instructions. Computer-readableinstructions can be stored on a tangible and/or non-transitorycomputer-readable medium, such as random access memory (RAM), read-onlymemory (ROM), volatile memory, nonvolatile memory, hard drives, compactdisc read-only memory (CD ROM), digital versatile disc (DVD) memory,flash drives, magnetic strip storage, semiconductor storage, opticaldisc storage, magneto-optical disc storage, flash memory, registers,caches, and/or any other storage medium. Alternatively, the methods canbe implemented in hardware components or combinations of hardware andsoftware such as, for example, ASICs and/or special purpose computers.

Embodiments have been described with reference to numerous specificdetails that can vary from implementation to implementation. Certainadaptations and modifications of the described embodiments can be made.Other embodiments can be apparent to those skilled in the art fromconsideration of the specification and practice of the embodimentsdisclosed herein. It is intended that the specification and examples beconsidered as exemplary only. It is also intended that the sequence ofsteps shown in figures are only for illustrative purposes and are notintended to be limited to any particular sequence of steps. As such,those skilled in the art can appreciate that these steps can beperformed in a different order while implementing the same method.

What is claimed is:
 1. A method comprising: (a) detecting, by a deviceintermediary to a plurality of clients and one or more servers, avariation of a level of quality of content being transmitted by the oneor more servers via one or more transactions to one or more clients ofthe plurality of clients based on at least a difference in a size of thecontent and an average size of content of the one or more transactions;(b) determining, by the device, one or more parameters for quality ofexperience associated with a client of the plurality of clients based onat least the detected variation of the level of quality of content; and(c) applying, by the device, one or more policies to transmission ofcontent to the client based on the one or more parameters.
 2. The methodof claim 1, wherein content comprises one of video or audio.
 3. Themethod of claim 1, wherein the client comprises a mobile terminal. 4.The method of claim 1, wherein the one or more servers comprises acontent server serving multimedia content.
 5. The method of claim 1,wherein (a) further comprises determining an average size of content oftransactions preceding the one or more transactions.
 6. The method ofclaim 1, wherein (a) further comprises determining an average size ofcontent of transactions subsequent to the one or more transactions. 7.The method of claim 1, wherein (a) further comprises determining adifference in size of content of a current transaction to an averagesize of content of preceding or subsequent transactions.
 8. The methodof claim 1, wherein (a) further comprises detecting the variation of thelevel of quality of content is greater than a predetermined threshold.9. The method of claim 1, wherein (c) further comprises dynamicallygenerating one or more rules of the one or more policies based at leaston monitoring of the one or more parameters.
 10. The method of claim 1,wherein (c) further comprises applying the one or more policies toimprove quality of experience associated with the client.
 11. A systemcomprising: a device comprising one or more processors, coupled tomemory and intermediary to a plurality of clients and one or moreservers; a detector of the device configured to detect a variation of alevel of quality of content being transmitted by the one or more serversvia one or more transactions to one or more clients of the plurality ofclients based on at least a difference in a size of the content and anaverage size of content of the one or more transactions; an estimator ofthe device configured to determine one or more parameters for quality ofexperience associated with a client of the plurality of clients based onat least the detected variation of the level of quality of content; anda traffic manager of the device configured to apply one or more policiesto transmission of content to the client based on the one or moreparameters.
 12. The system of claim 11, wherein content comprises one ofvideo or audio.
 13. The system of claim 11, wherein the client comprisesa mobile terminal.
 14. The system of claim 11, wherein the one or moreservers comprises a content server serving multimedia content.
 15. Thesystem of claim 11, wherein the detector is further configured todetermine an average size of content of transactions preceding the oneor more transactions.
 16. The system of claim 11, wherein the detectoris further configured to determine an average size of content oftransactions subsequent to the one or more transactions.
 17. The systemof claim 11, wherein the detector is further configured to determine adifference in size of content of a current transaction to an averagesize of content of preceding or subsequent transactions.
 18. The systemof claim 11, wherein the detector is further configured to detect thatthe variation of the level of quality of content is greater than apredetermined threshold.
 19. The system of claim 11, wherein the trafficmanager is further configured to dynamically generate one or more rulesof the one or more policies based at least on monitoring of the one ormore parameters.
 20. The system of claim 11, wherein the traffic manageris further configured apply the one or more policies to improve qualityof experience associated with the client.